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ABSTRACT 

The  Technology  Transfer  project  employs  a  spiral  development  process  to  enhance  the  functionality  and  autonomy  of 
mobile  systems  in  the  Joint  Robotics  Program  (JRP)  Robotic  Systems  Pool  (RSP).  The  approach  is  to  harvest  prior  and 
on-going  developments  that  address  the  technology  needs  identified  by  emergent  in-theatre  requirements  and  users  of  the 
RSP.  The  component  technologies  are  evaluated  on  a  transition  platform  to  identify  the  best  features  of  the  different 
approaches,  which  are  then  integrated  and  optimized  to  work  in  harmony  in  a  complete  solution.  The  result  is  an 
enabling  mechanism  that  continuously  capitalizes  on  state-of-the-art  results  from  the  research  environment  to  create  a 
standardized  solution  that  can  be  easily  transitioned  to  ongoing  development  programs.  This  paper  focuses  on  particular 
research  areas,  specifically  collision  avoidance,  simultaneous  localization  and  mapping  (SLAM),  and  target-following, 
and  describes  the  results  of  their  combined  integration  and  optimization  over  the  past  12  months. 
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1.  BACKGROUND 

The  objective  is  to  enhance  the  functionality  and  autonomy  of  mobile  robotic  systems  in  the  Joint  Robotics  Program 
(JRP)  Robotic  Systems  Pool  through  a  spiral-development  process  that  harvests  existing  component  technologies  for 
optimization.  The  Tactical  Mobile  Robot  (TMR)  program,  sponsored  by  the  Defense  Advanced  Research  Projects 
Agency  (DARPA),  was  transferred  to  Space  and  Naval  Warfare  Systems  Center,  San  Diego  (SSC  San  Diego)  at  the  end 
of  FY-02  to  facilitate  the  transition  of  TMR-funded  technology  into  ongoing  JRP  development  efforts.  SSC  San  Diego 
worked  with  a  variety  of  DARPA  contractors  to  extract  relevant  aspects  of  their  research  and  port  it  to  ongoing  projects 
and  systems  associated  with  the  JRP  Robotic  Systems  Pool.  The  continuing  search  for  supporting  technologies  has 
naturally  expanded  to  other  government  research  activities,  academia,  and  industry  to  further  foster  emergent 
technology-transfer  opportunities  (Figure  1). 

Accordingly,  the  JRP  Technology  Transfer  Program  has  teamed  with  a  number  of  organizations  with  similar  ambitions, 
such  as  the  Idaho  National  Laboratory  (INL),  to  assist  in  the  coordinated  development,  evaluation,  and  sharing  of 
robotics  technology.  INL  has  a  direct  interest  in  autonomous  robots  for  use  in  a  variety  of  DOE  missions,  including 
homeland  defense  and  critical  infrastructure  protection.  This  synergistic  teaming  between  SSS  San  Diego  and  INL  has 
two  obvious  advantages:  1)  The  INL  Robotics  Group,  with  similar  objectives  and  experience,  substantially  augments  the 
available  manpower  resources,  allowing  more  technology  options  to  be  evaluated;  and,  2)  active  DOE  involvement 
opens  up  another  major  conduit  for  exporting  the  results  into  relevant  government  applications. 

An  equally  important  objective  of  the  program  is  to  also  transition  relevant  technology  enhancements  into  the  private 
sector,  in  order  to  enhance  the  supporting  industrial  base.  The  National  Center  for  Defense  Robotics  (NCDR)  intends  to 
enter  into  one  or  more  CRADA  agreements  with  SSC  San  Diego  and  INL  (and  possibly  other  government  laboratories) 
to  facilitate  licensing  on  behalf  of  companies  and  other  commercial  entities  belonging  to  the  NCDRs  Agile  Robotics 
Alliance.  Alliance  companies  are  in  turn  expected  to  adapt,  further  develop,  and  integrate  such  technologies  into  current 
and  planned  unmanned  systems  they  are  engineering  and  producing  for  the  military,  as  well  as  their  targeted  commercial 
markets.  The  NCDR  expects  to  provide  funding  on  a  case-by-case  basis  to  partner  Alliance  members  with  the 
appropriate  government  laboratories  and  to  help  cover  the  up-front  assessment  costs  and/or  further  integration  work  that 
may  be  required. 
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1.1 


Technical  Challenges 


In  reexamining  Figure  1,  it  seems  readily  apparent  that  each  of  the  identified  players  is  making  a  synergistic  contribution 
to  the  collective  whole,  which  in  turn  should  be  rather  impressive  indeed  in  terms  of  autonomous  fimctionality  once  all 
the  individual  pieces  come  together.  In  reality,  however,  making  it  all  work  in  harmony  is  a  challenging  task.  The 
various  developers  each  have  their  own  preferences  and  constraints  in  terms  of  computer  architectures,  operating 
systems,  languages,  data  formats,  sensors,  embedded  hardware,  and  even  power  sources.  In  order  to  optimize  the 
various  component  technologies  into  a  single  system,  the  sensors  and  computational  hardware  to  support  the  software 
algorithms  must  first  be  integrated  onto  a  common  platform. 
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Figure  1.  The  Technology  Transfer  Program’s  technical  approach  is  to  integrate,  test,  and  optimize  existing  component 
technologies  into  a  single  solution  on  an  evaluation  platform,  miniaturize  for  man-portable  systems,  and  port  to  COTS  platforms. 


Problems  arise  when  attempting  to  transition  directly  to  man-portable-size  (i.e., 
less  than  80  pounds)  systems  in  two  main  areas:  1)  available  power  to  support  the 
required  sensors  and  their  associated  processors,  and,  2)  the  physical  size  of  this 
additional  hardware,  particularly  the  sensors.  Conventional  batteries  on  current 
man-portable  systems  last  only  about  4  hours,  and  these  rather  simplistic 
teleoperated  systems  are  nowhere  near  as  complex  or  power  hungry  in 
comparison.  For  example,  the  run-time  for  an  iRobot  PackBot  equipped  with 
JPL’s  stereo-  and  laser-based  obstacle-avoidance  systems  developed  under  the 
TMR  Program  dropped  fi*om  4  hours  to  a  mere  20  minutes  (Figure  2).  The  SICK 
laser  rangefinder  draws  20  watts  by  itself,  and  its  considerable  size  and  weight  (10 
pounds)  seriously  hinders  platform  mobility,  such  as  the  ability  to  climb  stairs. 
Accordingly,  a  strong  need  exists  to  provide  a  small,  light-weight,  non-contact 
scanning  range  sensor,  optimized  for  the  size/weight/power  restrictions  of  man- 


Figure2.  WU's  PackBot  equipped 
with  TMR-era  stereo  and  laser 
collision-avoidance  sensors. 


portable  robots.  Current  state-of-the  art  (i,e.,  the  SICK  ladar)  is  typically  geared  towards  automated  guided  vehicles 
used  in  factory  automation,  and  as  a  result  is  far  too  heavy  and  power-hungry  for  use  on  small  tactical  robots. 

Our  approach  for  technology  infusion  to  the  man-portable  systems  of  the  JRP  Robotic  Systems  Pool  is  therefore  a  three- 
step  process:  1)  integrate  and  optimize  the  component  technologies  to  work  in  harmony,  2)  scale  the  solution  down  in 
terms  of  power,  size,  and  weight;  and,  3)  infuse  the  results  into  commercial-off-the-shelf  systems.  Numerous  robotic 
test/evaluation  platforms  are  employed  at  both  SSC  San  Diego  and  INL  to  support  this  process. 

1.2  Test/Evaluation  Platforms 

The  original  test  and  evaluation  platform  was  ROBART  III  (Figure  3),  which  had 
the  required  size,  a  90-amp-hour  battery,  readily  available  source  code,  and 
extensive  diagnostics  needed  to  host  numerous  sensors  for  navigation  and  intruder 
detection.  It  is  currently  equipped  with  a  SICK  scanning  laser  rangefinder.  Sharp 
triangulation  ranging  sensors,  passive-infrared  (PIR)  motion  sensors,  Polaroid 
ultrasonic  rangefinders,  a  gyro-stabilized  magnetic  compass,  and  a  fiber-optic  rate 
gyro.  ROBART  Ill's  vision  system  includes  a  Visual  Stone  360-degree  omni¬ 
directional  camera  and  a  Canon  pan-tilt-zoom  (PTZ)  camera.  With  the  reserve 
capacity  to  host  even  more  sensors  and  their  associated  computational  hardware, 

ROBART  III  serves  as  an  optimal  laboratory  development  platform  for  step  one, 
integration  and  optimization  of  the  various  candidate  software  algorithms  under 
consideration.  It  also  has  a  non-lethal  Gatling-style  weapon  (on  right  shoulder 
pod)  to  support  higher-level  behavior  development  in  conjunction  with  the 
Warfighter 's  Associate  Concept.  ^ 

An  iRobot  Senior  was  similarly  used  at  SSC  San  Diego  in  FY-04  to  support 
outdoor  navigation  testing,  the  initial  thought  being  ROBART  III  would  address 
only  indoor  scenarios.  With  the  introduction  of  the  Warfighter's  Associate 
Concept  in  FY-05,  the  distinction  between  indoor  and  outdoor  platforms  was 
dropped,  since  soldiers  must  routinely  perform  in  both  environments,  and  any 
robot  intended  to  work  alongside  them  as  part  of  a  synergistic  team  must  do 
likewise.  Consequently,  the  ATRV  Senior  now  serves  as  the  primary  evaluation  platform  for  indoor  localization 
algorithms  because  of  its  incredibly  inaccurate  dead-reckoning  solution,  which  arises  from  its  wide  tires  and  skid-steer 
steering.  Its  larger  size  also  provides  the  surface  space  and  convenient  mounting  bars  to  install  additional  payloads, 
especially  those  of  significant  weight.  For  example,  an  ATRV  Senior  was  loaned  to  the  Applied  Research  Laboratory  at 
the  University  of  Texas  (UT)  at  Austin  for  further  testing  of  their  human-presence  sensor  on  a  moving  platform  (further 
discussed  in  section  2.1.4).  Another  ATRV  Senior  is  also  on  loan  to  UT’s  Robotics  Research  Group  to  support  mobile 
manipulation  development  for  vision-based  control  of  a  Barrett  arm  and  Barrett  hand. 

Once  the  appropriate  component  technologies  have  been  suitably  integrated  and  tested 
on  ROBART  ///and/or  the  Senior,  the  proven  autonomy/functionality  upgrade  is 

ported  over  to  the  Man  Portable  Robotic  System  (MPRS)  project  at  SSC  San  Diego  for 
miniaturization  on  the  URBOT  (Figure  4),  originally  developed  for  use  by  the  Army 
engineers  for  tunnel,  sewer,  cave,  and  urban  structure  reconnaissance  A  GPS 
waypoint  navigation  capability  was  developed  for  the  URBOT  in  2002  and  is  currently 
being  integrated  with  stereo-based  collision  avoidance  technologies  originally 
developed  by  the  Jet  Propulsion  Lab  under  TMR.^ 

INL  also  has  a  pool  of  various  iRobot  and  ActiveMedia  platforms  that  have  been  used 
to  optimize  Advanced  Robotic  Control  Architecture  (further  discussed  in  Section 
2.2)  for  cross-platform  compatibility. 


Figures.  ROBART  III 


2.  PROJECT  STATUS  UPDATE 


Recent  and  ongoing  military  actions  in  Afghanistan  and  Iraq  marked  the  first  time  robotic  systems  have  played  a 
meaningful  role  during  actual  combat  operations.  It  is  interesting  to  note  that  of  the  over  200  mobile  robotic  systems 
deployed,  all  of  them  are  strictly  tele-operated  with  no  autonomous  functionality.  As  a  consequence,  these  systems 
appear  organically  attractive  only  in  life-threatening  scenarios,  such  as  detection  of  chemical/biological/radiation 
hazards,  mines,  or  improvised  explosive  devices.  A  need  exists  for  significant  improvements  in  both  functionality  (i.e., 
perform  more  useful  tasks)  and  autonomy  (i.e.,  with  less  human  intervention)  to  increase  the  level  of  general  acceptance 
and,  hence,  the  number  of  units  deployed  by  the  user.  The  Technology  Transfer  Program  has  already  produced 
phenomenal  results  addressing  both  these  issues  through  optimization  of  component  technologies  integrated  in  FY-03 
and  FY-04.^  The  following  subsection  describes  our  current  status  to  develop  a  more  complete  solution  that  can 
significantly  enhance  warfighting  capabilities. 

2.1  Enhanced  Functionalities  and  Autonomy 

Improved  autonomous  navigation,  including  collision  avoidance,  mapping,  localization,  and  path  planning,  was  the 
primary  focus  through  FY-04,  and  a  system  incorporating  all  of  these  functionalities  has  been  optimized,  as  discussed  in 
the  subsections  below. 

2.1.1  Simultaneous  Localization  and  Mapping 

The  Consistent  Pose  Estimation  (CPE)  mapping  technology  was  developed  at  Stanford  Research  Institute  International 
(SRI).  CPE  efficiently  incorporates  new  laser  scan  information  into  a  growing  map  and  also  addresses  the  challenging 
problem  of  loop  closure,  how  to  optimally  register  laser  information  when  the  robot  returns  to  an  area  previously 
explored.  CPE  is  one  method  of  performing  Simultaneous  Localization  and  Mapping  {SLAM),  based  on  original  work 
by  Lu  and  Milios,'^  who  showed  that  information  from  the  robot’s  encoders  and  laser  sensors  could  be  represented  as  a 
network  of  probabilistic  constraints  linking  the  successive  poses  of  the  robot. 

SRI  has  implemented  and  further  developed  localization  algorithms  using  a  representation  of  the  robot’s  state  space 
based  on  Monte  Carlo  sampling.^  Introduced  in  1970,^  Monte  Carlo  Localization  (MCL)  methods  have  more  recently 
been  applied  with  good  results  in  the  fields  of  target  tracking,  computer  vision,  and  robot  localization^’  The  Monte 
Carlo  technique  inherits  the  benefits  of  previously  introduced  Markovian  probability-grid  approaches  for  position 
estimation^  and  provides  an  extremely  efficient  technique  for  mobile  robot  localization.  One  bottleneck  in  the  MCL 
algorithm  is  the  necessity  for  checking  the  posterior  probability  of  each  sample  against  the  map,  based  on  the  current 
laser  readings.  SRI  has  developed  an  efficient  method  for  performing  this  computation,  using  a  correlation  technique 
derived  from  computer  vision  algorithms.^ 

Follow-on  plans  are  to  integrate  and  evaluate  multi-robot  mapping  techniques  developed  under  DARPA’s  Software  for 
Distributed  Robotics  (SDR)  Program. 

2.1.2  Collision  Avoidance 

SRI’s  SLAM  algorithms  were  integrated  with  collision-avoidance  techniques  developed  by  INL  specifically  for  use  in 
dynamic  unknown  environments.  The  collision-avoidance  algorithms  take  a  behavior-based  approach  that  emphasizes  a 
tight  coupling  between  sensing  and  action,  with  each  of  the  sensors  contributing  to  an  array  of  robot-centric  regions  to 
which  the  robot  responds,  based  on  ftizzy-logic  rules  that  control  its  translational  and  rotational  velocities.  These  rules 
not  only  apply  to  each  individual  region,  but  can  be  triggered  by  combinations  and  patterns  found  within  the  array  of 
regions.  In  implementing  this  scheme  INL  uses  a  subsumption  architecture  similar  to  that  employed  on  ROB  ART  I, 
wherein  atomistic  behaviors  such  as  collision  avoidance  run  in  parallel  with,  but  can  be  subsumed  by,  other  reactive 
behaviors,  such  as  “maneuver-around”  and  “get  unstuck.”  Collision  avoidance  is  a  bottom-layer  behavior,  and  although 
it  underlies  many  different  reactive  and  deliberative  capabilities,  it  runs  independently. 

INL  has  also  incorporated  other  deliberative  behaviors  that  function  at  a  level  above  the  reactive  behaviors.  Once  the 
reactive  behaviors  are  “satisfied,”  the  deliberative  behaviors  may  take  control,  allowing  the  robot  to  exploit  the  map  in 
order  to  support  behaviors  such  as  area  search,  patrol  perimeter,  and  follow  route.  INL’s  guarded  motion  (i.e.,  reflexive 
teleoperation)  capabilities  employ  several  different  sensors  (i.e.,  scanning  laser,  infrared  triangulation,  sonar,  tactile, 
inertial,  and  tilt),  fiising  available  perceptual  data  into  regions  that  represent  the  ability  of  the  robot  to  move  safely  in  a 
given  direction.  The  algorithm  also  continuously  calculates  an  event  horizon  representing  the  last  possible  moment  for 


the  collision-avoidance  behavior  to  successfully  intervene  upon  goal-based  behaviors  at  the  current  speed.  By  calculating 
this  event  horizon  many  times  each  second,  the  robot  can  smoothly  scale  down  its  velocity  as  a  function  of  congestion 
without  necessarily  fully  impeding  motion.  When  a  full  stop  is  required,  use  of  the  event  horizon  ensures  that  the  robot 
comes  to  a  halt  at  the  same  distance  from  an  obstacle  regardless  of  its  initial  velocity. 


2.1.3  Global  Path  Planner 

In  early  FY-05,  SRI  added  a  stand-alone  global  path  planner  that  operates 
upon  the  map  generated  by  the  SLAM  algorithm.  This  gradient-based 
planner  uses  the  occupancy  grid  as  the  planning  environment  and  generates 
an  optimal  path  from  the  robot’s  current  position  to  any  desired  destination 
within  the  map.  An  example  path  trajectory  is  shown  in  Figure  5. 

The  global  path  planner  is  also  integrated  with  the  local  path  planner, 
allowing  the  robot  to  maneuver  from  a  known  environment  (area  that  has 
been  previously  explored  and  mapped)  to  an  unknown  environment  (area 
that  is  not  yet  mapped),  maximizing  both  efficiency  and  robustness.  For 
example,  when  a  destination  goal  sent  to  the  robot  is  located  outside  the 
current  map,  the  global  path  planner  will  plan  a  path  to  that  point  in  the  map 
closest  to  the  destination,  and  then  the  local  path  planner  will  seamlessly 
guide  the  robot  in  the  unknown  environment. 

2.1.4  Motion  Detection  on  the  Move 

Performing  intruder  detection  from  a  moving  platform  is  a  difficult  problem 
compared  to  sensing  intruders  with  a  static  sensor,  where  all  that  is  required 
is  to  detect  a  change  in  sensor  output.  An  intruder  must  be  moving  to  enter 
the  field-of-view  of  a  fixed  sensor,  hence  motion  detection  works  very  well. 

Detecting  a  human  from  a  moving  platform  is  considerably  more 
challenging  because  the  background  is  continuously  changing,  and 

averaging  or  background  subtraction  alone  will  be  unreliable  by  itself  Furthermore,  the  intruder  may  not  always  be 
moving. 


Figure  5.  Path  trajectory  (red  line  in  bottom 
map  display)  planned  by  the  global  path 
planner  to  reenter  the  building  through  the 
door  visible  in  the  right  top  comer  of  the 
video  feedback. 


SRI’s  SLAM  technology  was  leveraged  to  develop  a  change-detection-on-the-move  capability.  Once  a  map  is  built 
representing  the  monitored  area,  the  robot  uses  the  occupancy  grid  from  the  SLAM  algorithm  to  detect  changes  within 
the  environment.  The  location  of  the  change  is  sent  as  a  vector  to  a  video  camera  which  then  displays  the  “intruder”  to 
the  operator.  This  technology  is  being  leveraged  by  DTRA-fiinded  efforts  to  demonstrate  human  presence  detection  and 
assessment  (HPDA)  from  a  moving  robotic  platform.  Using  the  laser-based  SLAM  technology  will  allow  the  robot  to 
receive  range  information  to  objects  in  its  environment  and  determine  when  an  object  is  “new,”  meaning  it  had  not 
previously  been  detected.  The  output  vector  from  the  change-detection  algorithm  will  cue  a  passive  microwave  sensor 
being  developed  by  the  Applied  Research  Laboratory  at  the  University  of  Texas,  Austin  that  detects  signatures  unique  to 
humans  for  further  assessment. 


2.1.5  Vision-Based  Target  Identification/Following 

The  same  concept  of  using  a  vector  from  a  sensor  payload  to  cue  an  assessment  reaction  is  employed  by  the  target¬ 
following  behavior.  The  Sony  PTZ  Camera  onboard  the  ATRV  Senior  already  includes  color-blob  tracking  and  edge- 
detection  software.  The  camera  easily  tracks  a  pre-taught  object,  continuously  outputting  the  target’s  relative  bearing  to 
the  drive  subsystem,  enabling  the  robot  to  track  and  follow  a  pre-taught  target  while  avoiding  obstacles.  A  person¬ 
following  routine  for  the  Segway  Remote  Mobility  Platform  (RMP)  was  also  developed  at  SSC  San  Diego,  allowing  the 
RMP  to  continuously  follow  a  person  in  the  same  manner.  The  system  developed  for  the  Segway  RMP,  however,  uses 
hue-tracking  methods  that  are  more  independent  of  varying  lighting  conditions  than  color-blob  tracking.  The  system 
also  allows  for  gradual  changes  in  the  appearance  of  the  tracked  target,  sometimes  due  to  the  dramatically  differing 
characteristics  of  sunlight  versus  fluorescent  light  when  the  robot  is  moving  from  an  indoor  to  outdoor  environment. 

Distributed  Interactive  Video  Array  (DIVA)  technology  originally  developed  at  UCSD’s  Computer  Vision  and  Robotics 
Research  (CVRR)  Laboratory  has  been  ported  over  to  ROB  ART  III  in  FY-04  to  provide  an  advanced  vision  capability 
for  the  robot,  as  well  as  a  research  tool  to  develop  and  investigate  additional  vision-tracking  algorithms.  For  example, 


ROBART  III  currently  maintains  a  pre-taught  database  of  digital  color 
pictures  of  potential  targets  and  their  associated  “vulnerabilities.”  The 
vision  system  compares  these  target  templates  with  live  images  from  its 
incoming  video  stream,  ROBART  III  then  performs  a  two-stage  search- 
and-engage  algorithm/  wherein  the  vision  system  first  performs  a  wide- 
area  scan  for  a  pre-taught  class  of  objects,  tiien  cues  the  PTZ  camera  to 
zoom  in  and  search  for  specific  “vulnerabilities”  associated  with  that 
particular  target.  The  non-lethal  weapon  is  automatically  trained 
accordingly  with  the  aid  of  a  bore-sighted  targeting  laser,  and  then  fired 
under  operator  supervision,  using  a  color-correlation-matching  algorithm 
(see  Figure  6). 


Figure  6.  a)  Vision  System  and  targeting 

Future  plans  are  to  extend  the  person-following  routine  developed  on  the  laser  on  detected  vulnerability  (soda  can); 

Segway  RMP  by  fiising  data  from  visual  and  IR  cameras.  Fusing  color  data  b)  Can  is  relocated  and  tracked  in  real-time 

with  a  human’s  IR-signature  should  avoid  tracking  errors  caused  by  a  c)  Targeting  laser  servos  to  new  location; 

similarly-colored  background  or  people  wearing  similar  clothing.  ESfL  and  d)  Laser  now  relocated  on  new  target 

SSC  San  Diego  are  also  collaborating  on  a  system  to  detect  and  model  position,  ready  to  fire  weapon, 

doors,  doorknobs,  and  name  placards  for  robots  navigating  office 

environments.  This  capability  will  first  be  used  to  assist  the  robot  in  locating  individual  offices  and  later  assist  in 
grasping  and  manipulating  doorknobs  or  opening  doors. 


2.2  Common  Architecture 

To  facilitate  integration  and  ultimate  transfer  to  ongoing  programs,  our  approach  is  to  adapt  and  standardize  on  a 
reconfigurable  software  fi^mework  that  can  be  easily  ported  from  one  robotic  system  to  another.  Real  progress  will  not 
be  made  in  robotics  until  there  are  mutually  agreeable  standards  for  combining  different  component  technologies.  The 
huge  success  of  the  Internet,  for  instance,  was  only  made  possible  with  mutually  agreed  upon  standards  such  as  the 
TCP/IP  protocol. 


There  are  many  reasons  why  robotic  standardization  has  not  happened  sooner,  the  principle  factor  being  there  is  not  yet 
financial  motivation  for  such  standards.  Early  computer  makers  in  the  1950s  and  60s  did  not  standardize  their  products 
because  of  the  fear  of  competition  and  a  minimal  number  of  computer  users.  The  transition  of  information  technology 
from  an  engineering  solution  to  a  commodity  eventually  precipitated  the  need  for  standardization.  As  the  number  of 
users  grew,  interoperability  between  different  computer  products  became  more  critical.  Another  major  reason  that 
standardization  has  not  readily  taken  hold  is  that  there  are  varying  approaches  to  developing  autonomous  robotic 
technologies,  particularly  with  regard  to  behavior  arbitration,  knowledge  representation,  and  machine  learning. 

In  an  attempt  to  develop  cross-platform  compatibility,  a  few  de  facto  standards  have  more  recently  emerged  for  lower- 
level  robotic  control.  Many  of  these  are  commercial  software  packages,  such  as  ActivMedia’s  ARIA  and  iRoboFs 
Mobility  and  Aware.  There  are  also  some  open-source  software  standards  such  as  the  University  of  Southern 
California’s  Player/Stage  project,  Universite  de  Sherbrooke’s  MARIE,  and  Carnegie  Mellon  University’s  CARMEN.,  all 
of  which  address  the  lowest  levels  of  robotic  control.  They  attempt  to  provide  an  abstract  interface  to  the  physical 
hardware  and  can  also  be  used  to  provide  interfaces  to  robotic  algorithms. 


One  of  the  most  promising  open-source  efforts  for  standardization  of  low-level  control  we  have  investigated  is  the 
University  of  Southern  California  Player/Stage  project.  The  goal  of  the  Player  server  is  to  provide  a  TCP/IP  or  UDP/IP 
network  interface  for  robotic  sensors  and  actuators.  The  project’s  mailing  list  currently  has  hundreds  of  subscribers,  and 
the  software  has  been  downloaded  thousands  of  times.  Even  though  the  software  has  no  commercial  support,  the  active 
developer  community  makes  it  easy  to  fix  bugs.  And  while  the  software  is  not  perfect,  it  does  a  very  good  job  for  its 
intended  purpose. 

One  primary  advantage  to  adopting  Player  approach  is  that  it  is  fully  extendable,  making  it  very  easy  to  add  support  for 
new  hardware,  and  users  have  already  contributed  many  different  drivers  for  the  most  popular  robotic  hardware, 
peripherals,  sensors,  etc.  (There  are  currently  63  drivers  integrated  into  the  distribution,  not  including  custom  drivers  that 
users  have  not  published.)  The  usefulness  of  the  Player  software  increases  almost  exponentially  with  the  number  of 


drivers  available,  as  weeks  worth  of  programming  can  be  saved  by 
using  an  existing  driver  for  one’s  hardware.  Under  the  Technology 
Transfer  effort,  Player  drivers  were  created  for  ROBART  Ill's  custom 
sensors  and  actuators,  including  a  differential  drive  controller,  sonar 
rangefinders,  a  power  system  interface,  a  speech  synthesizer,  and  a 
non-lethal  weapon.  Some  improvements  were  contributed  back  to  the 
Player  project,  such  as  modifications  to  the  Canon  VC-C4R  driver. 
Despite  the  aforementioned  benefits  of  using  Player,  there  are  some 
disadvantages.  We  have  encountered  bugs  and  non-robust  driver  code 
while  developing  software  with  Player,  which  were  resolved  by 
manually  debugging  the  software  and  then  contributing  trouble  reports 
to  the  development  community.  At  this  time,  many  developers  still 
consider  Player  to  be  experimental,  as  further  development  is  needed 
for  robust  performance  on  fielded  platforms. 


Figure  7.  The  INL  control  architecture  has 
been  ported  to  their  pool  of  various  iRobot 
and  ActiveMedia  platforms  shown  above. 


Using  Player  as  the  low-level  interface  allows  a  logical  separation  of 
high-level  behaviors  and  low-level  control.  For  example,  INL’s 

Advanced  Robotic  Control  Architecture  has  been  optimized  to  be  independent  of  low-level  control.  The  Intelligence 
Kernel  implements  a  class  library  for  robotic  platforms,  sensors,  and  actuators,  allowing  any  type  of  low-level  interface 
to  be  used.  INL’s  control  architecture  currently  supports  Player,  Mobility,  ARIA,  and  some  other  proprietary  interfaces. 
(See  Figure  7.) 


INL’s  control  architecture  is  also  independent  of  the  robot’s  geometry 
and  sensor  suite.^^  The  entire  framework  is  object  oriented,  allowing 
all  software,  including  all  behaviors  and  associated  autonomous 
control,  to  be  easily  ported  to  a  variety  of  robotic  platforms  by  editing 
a  parameters  list  (i.e.,  for  robot  length,  width,  maximum  speed). 
Moreover,  the  system  allows  the  robot  to  recognize  what  sensors  it  has 
available  at  any  given  time  and  adjust  its  behavior  accordingly.  Such  a 
control  architecture  is  ideal  for  the  Technology  Transfer  Program,  as  it 
allows  easy  porting  and  testing  of  advanced  behavior  functionalities  on 
multiple  platforms. 

Another  notable  standardization  effort  is  the  Joint  Architecture  for 
Unmanned  Systems  (JAUS),  a  JRP  initiative  to  define  and  implement 
an  upper-level  architecture  design  for  a  common  interface  to  a  variety 
of  unmanned  vehicles,  sensors,  and  munitions.^^  JAUS  is  component- 
based,  specifying  data  formats  and  methods  of  communication  among 
computing  nodes.  The  JAUS  Working  Group  (made  up  of  members 
from  the  U.S.  government,  industry  and  academia)  defines  methods  for 
message  passing  and  standards  for  component  behaviors  in  order  to  be 
independent  of  technology,  computer  hardware,  operator  use,  vehicle 
platform,  and  mission.  SSC  San  Diego  is  an  active  member  of  the 
Working  Group  and  has  developed  a  JAUS  interface  for  INL’s  Robotic 
Control  Architecture  so  that  any  JAUS-compliant  Operator  Control 
Unit  (OCU)  can  control  any  robot  using  the  INL  onboard  architecture. 

2.3  Augmented  Virtuality 

Controlling  or  supervising  advanced  robotic  behaviors  requires  a 
suitable  high-level  interface  human-robot  for  mixed-initiative  control 
and  efficient  tasking.  In  an  attempt  to  develop  a  shared  workspace  for 
effective  command  and  control,  an  augmented  virtuality  interface, 
based  on  underlying  technologies  developed  at  Brigham  Young 
University,  is  being  developed  that  can  link  additional  sensor 
information  to  the  robot’s  world  model. 


Figure  8.  The  operator’s  perspective  of  the 
environment  is  adjustable  by  changing  the 
zoom,  pitch,  and  yaw  of  the  2  V2-D  interface. 


The  world  representation  is  not  true  3-D,  but  obtained  by  arbitrarily  growing  the  laser-generated  2-D  SLAM  map  some 
finite  vertical  distance,  and  consequently  referred  to  as  2^2-0.  The  zoom,  pitch,  and  yaw  of  the  2V2-D  interface  can  be 
changed,  allowing  the  operator’s  perspective  to  transition  from  a  bird’s-eye  view,  where  the  entire  environment  can  be 
seen  at  once,  to  a  first-person  perspective  (Figure  8).  This  approach  has  been  shown  to  provide  improved  situational 
awareness  for  driving  the  robot  and  understanding  the  local  environment  (especially  in  tight  spaces)  than  actual  real-time 
video  imagery.  The  augmented  virtuality  interface  can  also  be  supported  with  a  low-bandwidth  data  link  (e.g.,  900- 
MHz  serial  RF  link  at  9600  baud),  facilitating  communication  through  thick  concrete  and  rebar  in  urban  environments 
(see  Figure  9). 


Figure  9.  2^2-0  interface  obtained  from  the  2-D  SLAM  map  of  Battery  Woodward,  an 
underground  World  War  II  bunker  at  SSC  San  Diego  built  with  thick  concrete  walls. 

We  “augment”  the  2y2-D  interface  with  even  more  virtual  information  derived  from  on-board  sensor  readings  and/or 
operator  input.  For  example,  alarm  readings  from  the  CHARS  (chemical,  gas,  radiological  sensor)  application  payload 
developed  at  SSC  San  Diego,^^  could  be  “tagged”  with  an  appropriate  icon  in  the  augmented-virtuality  layer  of  the 
SLAM  world  model.  Likewise,  the  robot’s  vision  camera  can  be  treated  as  another  onboard  sensor  that  can  contribute 
snippets  of  video  and/or  still  imagery  (see  Figure  10  [better  picture])  of  conditions  encountered  at  specific  locations, 
which  similarly  be  linked  to  this  same  tag,  providing  both  virtual  and  real  elements  for  later  viewing.  Any  data  from 
onboard  sensors  can  be  time-  and  position-stamped  with  respect  to  the  virtual  model,  making  registration  (at  least  of 
robot-collected  data)  rather  simplistic. 

Future  plans  are  to  integrate  the  augmented  virtuality  interface  with  a 
geographic  information  system  (GIS)  system,  in  order  to  further  support 
the  need  to  bring  all  sources  of  data  together  in  a  map-like  environment  to 
be  visually  plotted  and  analyzed.  Such  a  tool  can  be  used  in  multiple 
DOD  and  DOE  domains,  such  as  security,  force  protection,  intelligence, 
command  and  control,  peacekeeping  operations,  and  facilities 
management. 

As  a  result  of  widespread  adoption  of  global-positioning-system  (GPS) 
technology,  remote  sensing,  and  surveying,  the  availability  of  spatial  data 
is  growing  fast.  Furthermore,  advancing  spatial  technology  makes  it 
possible  to  store  and  manage  all  of  this  data  in  a  standardized  database 
management  system  (DBMS).  An  inherent  advantage  of  this  approach  is 
the  ability  to  augment  existing  published  geo-spatial  data  such  as  aerial 


FFFI 


Figure  10.  Screenshot  of  the  virtual  model 
ftised  with  real-time  video  images  from  an 
ATRV  exploring  INL  office  space. 


photos,  vector  maps,  and  terrain  elevation  data  to  meet  the  needs  of  a  specific 
mission.  For  example,  a  SLAM-equipped  robot  could  autonomously  map  out 
an  unknown  bunker,  and  then  upload  the  2-D  (or  even  2^2-0)  model  of  the 
bunker  to  a  GIS  database.  Once  the  data  is  imported  into  the  GIS  system,  it 
can  be  seamlessly  viewed  with  other  data  sources,  as  well  as  analyzed  using 
other  GIS  applications.  The  highest  resolution  elevation  data  commonly 
available  is  Digital  Terrain  Elevation  Data  (DTED)  Level  2,  which  provides 
30-meter  resolution.  Using  differential  GPS,  an  unmanned  vehicle  could 
explore  an  area  and  develop  extremely  high-resolution  elevation  data  that 
could  be  used  for  line-of-sight  calculations,  radio  coverage  analysis,  or  other 
applications. 

SSC  San  Diego  has  been  approved  to  use  the  Commercial  Joint  Mapping 
Toolkit  (C/JMTK).  C/JMTK  is  a  major  acquisition  program  contracted  to 
TASC,  a  business  unit  of  Northrop  Grumman  Information  Technology  (IT)  by 
the  National  Imagery  and  Mapping  Agency  (NIMA)  to  provide  a 
comprehensive  standardized  commercial  toolkit  of  software  components  for 
the  capture,  management,  dissemination,  analysis  and  visualization  of 
geographic  and  related  information  for  all  DOD  battle-space  applications. 
C/JMTK  is  based  on  the  Environmental  Systems  Research  Institute  (ESRI) 
family  of  software  products  called  ArcGIS,  which  is  built  on  a  common 
architecture  that  forms  a  multi-user  GIS.  SSC  San  Diego  is  developing  a  3-D 
command-and-control  system  based  on  the  C/JMTK  framework  that  will 
provide  unprecedented  visualization  capabilities.  By  adopting  the  GIS  model, 
this  system  will  be  able  to  easily  share  information  with  many  other  military 
command-and-control  systems,  such  as  the  Global  Command  and  Control 
System-Joint  (GCCS-J). 

C/JMTK  includes  ArcGIS  software  components  to  serve  as  the  DBMS, 
Internet  server,  a  Spatial  Analyst,  a  3D  Analyst,  and  a  Military  Overlay  Editor 
(MOLE).  As  unknown  interior  structures  are  explored  and  mapped,  their 
corresponding  augmented  virtuality  interfaces  can  be  added  to  the  DBMS  and 
made  available  over  the  Internet.  With  the  3-D  Analyst,  the  tools  for 
providing  3-D  visualization  and  analysis  are  present  to  incorporate  the  2  Vi-D 
augmented  virtual  representation  of  the  interior  structures.  The  Military 
Overlay  Editor  (MOLE)  is  a  symbol  generator  and  editor  to  create  and 
position  unit  symbols  against  a  background  of  geographic  data.  The  MOLE 
software  component  can  be  used  to  create  new  standardized  symbols  used  to 
tag  locations  in  the  augmented  virtuality  interfaces  of  different  data  gathered 
from  various  sensor  payloads. 

3.  PLANNED  DEMONSTRATION 

Future  plans  are  to  demonstrate  the  autonomous  deployment  and  collaborative 
behaviors  of  multiple  robots  in  a  MOUT  environment.  The  individual  phases 
and  supporting  technology  areas  needed  are  already  demonstratable  at  SSC 
San  Diego  as  stand-alone  systems,  such  as  GPS  waypoint  navigation, 
deployment  of  marsupial  robots,  and  mapping  of  interior  structures.  The 
planned  FY-05  demonstration  will  illustrate  the  integration  and  further 
development  of  these  stand-alone  systems  as  follows. 

An  optimal  path  of  approach  is  first  created  by  the  operator  selecting  a  series 
of  waypoints  on  the  Multi-robot  Operator  Control  Unit  (MOCU)  map 
display,  based  on  recently  downloaded  overhead  imagery  as  shown  in  Figure 
11,  The  delivery  vehicle  is  then  dispatched  to  the  insertion  point  by  executing 


Figure  11.  MOCU  display  showing 
planned  route  to  insertion  point  for  the 
marsupial-carrier  configuration  (Figure 
12k 


Figure  12.  MDARS-Expeditionary 
robotic  vehicle  is  shown  deploying  an 
Urban  Robot  (URBOT)  at  the  insertion 
point,  in  preparation  for  the  building- 
penetration  phase  of  the  demo. 


Figure  13.  A  special  “information- 
available”  icon  appears  on  the  MOCU 
display  indicating  the  robot  is  uploading 
data  for  the  associated  interior  structure 
(i.e.,  cave,  bunker,  building). 


autonomous  waypoint  navigation  and  collision  avoidance  under  remote 
operator  supervision.  This  diesel-powered  vehicle  is  equipped  with  a 
marsupial  carrier  for  a  battery-powered  man-portable  robot,  such  as  the  SSC 
San  Diego  URBOT,  the  Foster-Miller  Talon,  or  the  iRobot  Packbot,  Upon 
arrival  at  the  insertion  point,  the  tracked  man-portable  robot  (in  this  case,  an 
URBOT  as  shown  in  Figure  12)  descends  from  the  marsupial  carrier  and 
moves  toward  the  building  entrance.  The  delivery  vehicle  can  remain  on 
station  as  a  communications  relay,  provide  defensive  cover,  or  be  otherwise 
redeployed  as  desired  by  the  operator. 

The  man-portable  robot  seamlessly  transitions  to  SLAM  navigational  mode 
upon  entering  the  building,  as  GPS  will  immediately  cease  to  function  due  to 
satellite  occlusion.  In  all  probability,  the  RF  link  will  also  be  lost  as  the  robot 
penetrates  deeper  into  the  interior  structure,  but  this  does  not  represent  a 
problem  either,  since  maintaining  a  real-time  data  link  is  not  required.  The 
SLAM-enabled  robot  will  continue  to  execute  a  complete  search  of  the  bunker 
in  autonomous  fashion,  augmenting  its  evolving  world  model  with  appropriate 
sensor  data,  still  imagery,  and  video  clips  as  appropriate.  If  a  valid 
communications  path  is  available  at  any  point,  a  copy  of  the  augmented 
virtuality  model  is  passed  back  to  the  operator  control  unit,  whereupon  an 
“information  available”  icon  appears  on  the  building  being  explored,  as  shown 
in  Figure  13.  Clicking  on  the  icon  will  bring  up  the  augmented  virtuality 
display  of  the  interior  structure  (Figure  14). 


Figure  14.  Clicking  on  the 
“information-available”  icon  for  a  so- 
designated  structure  brings  up  the 
augmented-virtuality  display  of  the 
structure’s  interior  being  mapped  by  the 
robot. 


Upon  completion  of  the  structure  sweep,  the  robot  plans  a  path  back  to  the  entrance  and  exits  the  bunker,  switching  back 
to  GPS  navigation,  and  re-establishing  RF  communications  if  not  already  acquired.  The  robot  can  then  be  recovered  by 
the  marsupial  carrier  for  recharging  and  subsequent  redeployment,  or  instructed  to  search  and  map  another  nearby 
structure. 


4.  CONCLUSION 

Military  robotic  capabilities  are  being  rapidly  expanded  through  the  efforts  of  the  Technology  Transfer  program 
managed  by  the  Space  and  Naval  Warfare  Systems  Center,  San  Diego.  To  address  the  need  for  advanced  functionality 
and  autonomy  based  on  feedback  from  the  JRP  Robotics  Systems  Pool  users,  this  program  has  combined  the 
development  efforts  of  many  key  players  in  the  robotics  community  for  transition  to  COTS  systems.  Efforts  in  FY-02 
thru  FY-05  have  produced  an  optimized  system  for  advanced  navigation  behaviors  (including  collision  avoidance, 
mapping,  localization,  path  planning,  and  target  identification/tracking),  on  a  cross-platform  compatible  software 
framework.  An  augmented  virtuality  interface  is  also  being  developed  to  combine  the  enhanced  functionalities  with  a 
more  intuitive  and  informative  user  interface,  increasing  the  warfighter’s  situational  awareness  and  safety.  The 
Technology  Transfer  program,  thus,  serves  as  enabling  mechanism  that  continuously  capitalizes  on  state-of-the-art 
contributions  from  the  research  environment  to  create  a  standardized  solution  that  can  be  easily  transitioned  to  ongoing 
development  programs. 
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